INTRODUCTION:

Retinex theory deals with compensation for illumination effects in images. The primary goal is to decompose a given image S into two different images, the reflectance image R, and the illumination image L, such that, at each point (x, y) in the image domain, S(x, y) = R(x, y) · L(x, y). The benefits of such decomposition include the possibility of removing illumination effects of back/front lighting, enhancing shots that include spatially varying illumination such as images that contain indoor and outdoor zones, and correcting the colors in images by removing illumination induced color shifts. Recovering the illumination from a given image is known to be a mathematically ill-posed problem. The Retinex methodology was motivated by Land’s landmark research of the human visual system [11]. Through his experiments it was shown that our visual system is able to practically recognize and match colors under a wide range of different illuminations, a property that is commonly referred to as the Color Constancy Phenomenon. As a matter of fact, Land’s findings indicated that even when retinal sensory signals coming from different color patches under different illuminations are identical, subjects were able to name the surface reflectance color [11]. The ability to extract the illumination image is sufficient but not necessary to achieve this property.

A first step taken by most algorithms is the conversion to the logarithmic domain by s = log S, l = log L, r = logR, and thereby s = l + r. This step is motivated both mathematically, preferring additions over multiplications, and physiologically, referring to the sensitivity of our visual system [11]. If color appearance is to be a useful feature in identifying an object, then color appearance must remain roughly constant when the object is viewed in different contexts. People maintain object color appearance despite variation in the color of nearby objects and despite variation in the spectral power distribution of the ambient light.1-4 historically, changes in the color appearance of an object caused by variation in the surface reflectance functions of surrounding objects have been called simultaneous contrast, whereas changes in color appearance caused by variation in the spectral power distribution of the ambient light have been called failures of color constancy. Both of these effects reduce the usefulness of color appearance as a feature for identifying objects. In our view, color constancy should be defined as the maintenance of color appearance despite variation in the color of nearby objects and despite variation in the spectral power distribution of the ambient light. Although human color vision does not maintain perfect color constancy, human performance is better than that of any currently available man-made systems. The fundamental difficulty in designing a color constant system arises because the light in the visual image confounds two factors: the spectral power distribution of the ambient light and the surface reflectance of the objects in the scene. The problem of separating these two factors cannot be solved for all possible viewing conditions. For example, if there is only a single unknown object in the image illuminated by an unknown light source, no algorithm can correctly determine the surface reflectance of the object. It follows that all color constant algorithms must use information obtained from light reflected from several different objects in the scene. We therefore think it natural to broaden the definition of color constancy to include preservation of appearance across variation in the surface reflectance of nearby objects.

Image Enhancement:

The image enhancement based on color and contras enhancement so before dealing with image enhancement we deal with color and contras enhancement. The goal of color and contrast enhancement in general is to provide a more appealing image or video with vivid colors and clarity of details. These enhancements are intimately related to different attributes of visual sensation.

The objective of contrast enhancement is to increase the visibility of details that may be obscured by deficient global and local lightness. The goal of color enhancement can be either to increase the colorfulness, or to increase the saturation. Increasing the lightness can give a perception of increased colorfulness, however in this case perceived saturation reduces for a given chroma. On the other hand, perceived saturation can be increased by increasing chroma or reducing lightness, or both. If chroma is increased moderately while slightly reducing the lightness, both saturation and colorfulness in an image can be enhanced. This method is also likely to avoid out-of-gamut or unrealizable colors. For any color enhancement method based on retinex theory, the main weakness lies in the fact that no direct interdependence is assumed between the luminance and chrominance data. Even though methods like color restoration described above were proposed as an extension, it is very difficult to maintain the relationship between lightness and chroma. Note that nonlinear processing is performed on the luminance data in a color space where luminance and chrominance data are not necessarily decoupled. Algorithms based on retinex theory are in many ways simply lightness adjustment and/or local contrast enhancement algorithms. Further, retinex based methods are computationally expensive, making them difficult to implement in commercial imaging devices.

Color constancy algorithms:

There are various color constancy algorithms exists like General transformation based approaches, Diagonal transformation based approaches, Gray World and Scale By Max approaches, Retinex approaches, Gamut mapping approaches, Statistical approaches, Machine learning approaches.

General transformation based Approaches:

In early 1980's transform based approaches were introduced. Most authors consider the transformation to be a linear map of 3 x 3 matrices. Gershon et al. [33] proposed an algorithm to solve for the transformation, based on three assumptions: (i) both the illumination and the surface reflectance spectra can be modeled using small dimensional basis sets, (ii) the average surface reflectance in every Mondrian patch is the same, and (iii) the illumination is uniform. The algorithm solved for the illuminant first and then estimated the transformation. However, the algorithm showed poor performance because the second assumption varied significantly and it is very difficult to always maintain uniform illumination. Maloney et al. [42, 43] proposed a 3-2 algorithm to solve for the limitations of [33] by modifying the assumptions. They made two further assumptions: (i) if there are n sensors, then the dimensionality of the illuminant is less than or equal to n and (ii) the illumination is locally uniform. These assumptions suggest that pseudo-inverse can be applied to solve for color constancy, if the surface reflectance are two dimensional. Unfortunately the surface reflectances have higher dimension. Forsyth [24] extended [42, 43] these algorithms, MWEXT (Maloney-Wandell EXTension), to obtain a set of plausible mappings instead of a unique mapping.

Diagonal transformation based approaches:

The color constancy algorithms are based on diagonal matrix transformation. In this case, color constancy is obtained, by simply taking the dot product of diagonal matrix and the image matrix obtained under unknown illumination. This is equivalent to independently scaling each channel by a factor. West et al. [58] showed that von Kries hypothesis that chromatic adaptation is a central mechanism for color constancy is based on the diagonal matrix transformation. Barnard et al. [5] and Finlayson et al. [23], proposed a sensor sharpening method to improve the performance of the color constancy algorithms based on diagonal matrix transformation. The idea of sensor sharpening is to map the data by a linear transform into a new space where diagonal models are more reliable. The final result is then mapped back to the original RGB space by taking the inverse transformation. The performance of the color constancy algorithms depending on diagonal transformation is improved by spectral sharpening in terms of low root mean square error.

Gray World and Scale by Max approaches:

Gray World [12] and Scale by Max algorithms [2] are regarded as simple algorithms on the basis of simplicity of their implementation. They are still used as a benchmark for comparison when it comes to algorithmic approach to color constancy. The gray world algorithm is one of the oldest and the simplest color constancy algorithms. It is based on the assumption that the color in each sensor channel averages to gray over the entire image.

The gray world algorithm estimate the deviation from the assumptions and is given by a simple expression, lr = mean(ER); lg = mean(EG); lb = mean(EB)

where lr, lg, lb are the mean value in each channel respectively and ER, EG, EB are individual image channels. In the scale by max algorithm, the estimate of the illuminant is obtained by measuring the maximum of the responses in each channel. The estimation formulation is very similar to that of the GW algorithm in equation (2), except for the fact that mean is replaced by the maximum of the sensor responses in each channel. It is a subset of the Bayesian approach under the assumption that the reflectance is independent and uniform [49]. The presence of specularities in the images means that the maximum reflectance is greater than pure white and it leads to incorrect illuminant estimation. Alternatively, these specularities can be used to measure the illuminant chromaticity.

Gamut approaches:

The concept of gamut approach is based on the work of Forsyth [24] presented in the early 1990's. It can also be referred to as a constraint based approach because color constancy is achieved by imposing constraints on the reflectance and/or the illuminant of the scene. It also imposes hard constraints on the range of occurrence of the illuminant [21, 24]. The implementation of gamut algorithms requires the knowledge of the canonical illuminants. The initial approach was proposed in the RGB color space, so it is also referred to as 3D gamut mapping algorithm. It is a two step approach. In the first step, two possible gamuts are obtained namely, the canonical gamut and the image gamut. The canonical gamut is obtained by taking the set of all possible (R; G;B) values due to surface reflectance under canonical illuminant. The choice of the canonical illuminant is arbitrary. Similarly, the image gamut is obtained by taking the set of all possible (R; G;B) values due to surface reectance under unknown illumination. Both gamuts are convex and are represented by the convex hull. In the second step, under the diagonal assumptions, both convex hulls are mapped. The image gamut is mapped onto the canonical gamut using a linear mapping procedure developed by Forsyth, [24] and called Maloney{Wandell EXTension. MWEXT required both the surface Reflectance and illuminants to be selected from a inite dimensional space.. This posed some limitation on the MWEXT. Forsyth suggested an algorithm CRULE (based on coefficient rule) to solve for the MWEXT limitations. A heuristic approach was adopted to select a single diagonal mapping from the set of plausible mappings [24]. Finlayson [21] proposed a modification to Forsyth's theory [24] in his work on gamut mapping color constancy in 2D space. Both [21, 24] used the same heuristic approach for the selection of a single mapping matrix. Barnard [9] suggested a mapping selection method based on averaging the set of feasible mappings in both the chromaticity space and the RGB space. This method is based on the assumption that all illuminants and their corresponding mappings are equally probable. Under such assumption, the mean or the expected value is used for the selection of the single mapping. However, in the 2D perspective method [21], unwanted distortion affected the mapping sets; thereby suggesting that the 2D mean estimation for the selection of a single mapping is biased in the chromaticity space. Therefore, Finlayson et al. [20] suggested a mean estimation from the reconstructed 3D maps. Finlayson et al. [19] also proposed angular error and median based mapping selection.

Statistical approaches:

Color constancy algorithms discussed under this classification are often based on the basic statistical assumption that the probability distribution of the data is Gaussian. Maximum likelihood is used as the parameter estimator [38]. However, there are some algorithms that applied different probability distributions [49] and parameter estimators [10, 25, 50]. Freeman et al. [25] applied Bayesian theory to color constancy. They provided an insight on how to use all the information about the illuminant that is contained in the sensor response, including the information used by the gray world, subspace and physical reliability algorithms. The algorithms [40, 22] assumed a priori knowledge on the illumination distribution. The a priori knowledge on the occurrence of the illumination can be assumed to be uniform, i.e., probability of occurrence of all the illuminants is equal. This assumption is fair, if the range of occurrences of the illuminant is not known. Alternatively, if the range of occurrence of the illuminant is known, then apriori information on the illuminant can be estimated from a specified set of images within that range. In their work on Bayesian based color constancy Brainard et al. [10] and Freeman et al. proposed a bilinear modeling technique to estimate the spectral distribution from the statistical information of the illuminants. They developed a multi-sensor Bayesian technique for color constancy by sequentially acquiring measurements from independent sensors. They proposed a simple bilinear diagonal color model and an iterative linear update method based on maximum a posteriori (MAP) estimation technique. They assumed a multivariate Gaussian distribution and the dichromatic reflectance model which is limited.

Machine learning approaches:

Machine learning algorithms are data based approaches. These algorithms involve two stages, training and testing. In the training stage, the algorithm learns the functional association between the input and the output data. Based on the learning, they predict the output of previously unseen data in the testing stage. So the sample dataset and training algorithms used in the training stage pretty much define these approaches. Therefore, pre- processing of the dataset is very important in order to avoid any undesirable prediction due to outliers(s). Initial learning approaches to color constancy were based on neural networks. Cardei et al. [13] and Funt et al. [29] in their work on color constancy proposed a multilayer perceptron (MLP) feedforward neural network based approach in the chromaticity spaces. The proposed network architecture consisted of 3600 input nodes, 400 neurons in the first hidden layer, 40 neurons in the second hidden layer and 2 output neurons. They experimented with both synthetic and real dataset. In the case of real images, significantly large numbers of images were required to train the network. Due to the practical limitation of collecting a large number of images, Funt et al. [29] adopted a statistical approach known as bootstrapping to generate a large number of training images from a small sample of real images. They showed that neural networks achieved better color constancy than color by correlation [13]. Ebner [17] proposed a neural network performing parallel algorithm in the RGB color space. Moore et al. [44] addressed the issue of multiple illuminations in their application of neural network for color constancy. Nayak et al. [45] proposed a neural network approach in the RGB space to achieve color correction for skin tracking. Stanikunas et al. [54] performed an investigation of color constancy using neural network and compared it to the human vision system. They concluded that background color information is important to achieve human equivalent color constancy in machine vision systems. Apart from neural networks, there are other machine learning algorithms that have also been applied to achieve illumination invariance description of a surface reflectance. Huang et al. [34] used an adaptive fuzzy based method called fuzzy associated memory (FAM) to recognize color objects in complex background and varying illumination conditions. Funt et al. [26] showed how Vapnik's support vector machines [57] can be applied to estimate the illumination chromaticity and also by incorporating the brightness information. They provided discussion on polynomial and radial basis function kernels. They showed that under controlled (laboratory) conditions support vector machines performs better than neural networks and color by correlation.

Retinex Theory:

The problem of retinex correction in image processing is one which has significant implications for the field of image processing. The problem arises in the field of computer vision where a captured image may not align with how reality appears. A popular example of this phenomenon is Adelson's checker shadow illusion (Fig.1). The human brain perceives an image and light sources differently from how photo-sensors and thus cameras may perceive them by sub-consciously correcting brightness, contrast, and colors or removing noise, shadows, glare, or reflections. The purpose of the models we investigate are those which attempt to alter an image so that it is representative of human perception. For example, a properly retinex corrected version of an image may have colors adjusted to their perceived values while correcting brightness and removing effects of lighting. Therefore we seek a model which is able to correct for one or more of these phenomena of human perception.

Fig 2: Box A appears to be darker than B even though both are the same shade.

Different retinex theory:

1. Single Scale Retinex (SSR)

2. Multi-Scale Retinex (MSR)

3. Multi-scale Retinex with color Restoration (MSRCR)

4. MSRCR with ‘canonical’gain/offset

CONCLUSION:

Image enhancement plays very crucial role in image processing like biometric pattern recognition like face, finger print, Iris etc. In Most cases the quality of an image is good but when quality of an image is degreed due to some external source like noise, etc than image enhancement is necessary step to reconstruct the image for further operation. So the mechanism of Retinex theory is used to enhance the image based on lighting condition . In this paper I focus some color constancy techniques and these techniques are comparatively gives low performance as compare to retinex theory.

REFERENCES:

1. H. von Helmholtz, Handbuch der Physiologischen Optik, 2^nd ed. (Voss, Hamburg, 1896).

2. H. Helson, "Fundamental problems in color vision. I. The principle governing changes in hue, saturation and lightness of nonselective samples in chromatic illumination," J. Exp. Psychol. 23, 439-476 (1938).

3. D. B. Judd, "Hue saturation and lightness of surface colors with chromatic illumination," J. Opt. Soc. Am. 30, 2-32 (1940).

4. E. H. Land and J. J. McCann, "Lightness and retinex theory," J. Opt. Soc. Am. 61, 1-11 (1971).

5. M. H. Brill, "A device performing illuminant invariant assessment of chromatic relations," J. Theor. Biol. 71,473-478 (1978).

6. G. Buchsbaum, "A spatial processor model for object color perception," J. Franklin Inst. 310, 1-26 (1980).

7. L. T. Maloney and B. A. Wandell, "Color constancy: a method for recovering surface spectral reflectance," J. Opt. Soc. Am. A 3, 29-33 (1986).

8. E. H. Land, "Recent advances in retinex theory and some implications for cortical computations: color vision and the natural image," Proc. Nat. Acad. Sci. USA 80, 5163-5169 (1983).

9. E. H. Land, D. H. Hubel, M. S. Livingston, S. H. Hollis, and M. M. Burns, "Colour-generating interactions across the corpus callosum," Nature 303, 616-618 (1983).

10. J. Frankle and J. J. McCann, "Method and apparatus for lightness imaging," United States Patent No. 4,384,336, 1983.

11. J. J. McCann and K. Houston, "Calculating color sensations from arrays of physical stimuli," IEEE Trans. Systems Man Cybern. SMC-13, 1000-1007 (1983).

12. D. J. Jobson, Z. Rahman, and G. A. Woodell, “A Multiscale Retinex for Bridging the Gap Between Color Images and the Human Observation of Scenes”, IEEE Trans. on Image Proc., Vol. 6, 1997.

13. R. L. Lagendijk and J. Biemond, Iterative Identification and Restoration of Images, Kluwer Academic Publishing, Boston, Massachusetts, 1991.

14. E. H. Land, “The Retinex Theory of Color Vision”, Sci. Amer., Vol. 237, pp. 108-128, 1977.

15. E. H. Land, “Recent Advances in the Retinex Theory and Some Implications for Cortical Computations: Color Vision and the Natural Image”, Proc. Nat. Acad. Sci. USA, Vol. 80, pp. 5163–5169, 1983.

16. E. H. Land, “An Alternative Technique for the Computation of the Designator in the Retinex Theory of Color Vision”, Proc. Nat. Acad. Sci. USA, Vol. 83, pp. 3078–3080, 1986.

17. E. H. Land, and J. J. McCann, “Lightness and the Retinex Theory”, J. Opt. Soc. Am., Vol. 61, pp. 1–11, 1971.

18. D. G. Luenberger, Linear and Non-Linear Programming, Addison-Wesley Pub., Menlo-Park, California, Second Edition, 1987.

19. J. Marroquin, J. Mitter and T. Poggio, ”Probabilistic solution for ill-posed problems in computational vision”, J. of the American Statistical Assos., Vol. 82, pp. 76-89, 1987.

20. A. V. Oppenheim and R. W. Schafer, Digital Signal Processing, Prentice Hall: New Jersey, 1975.

21. A. Papoulis, Probability, Random Variables, and Stochastic Processes, third Edition, pp. 345–346, McGraw-Hill, 1991.

22. D. Terzopoulos, “Image Analysis Using Multigrid Relaxation Methods”, IEEE Trans. on PAMI, Vol. 8, 129–139, 1986.

23. R. Kimmel, M. Elad, D. Shaked, R. Keshet, and I. Sobel, “A Variational Framework for Retinex”, Hewelett Packard Technical Report HPL-1999-151, June 1999.

24. T. G. Stockham Jr, “Image Processing in the Context of a Visual Model” Proc. of the IEEE, Vol. 60, 828–842, 1972.

Received on 11.03.2011 Accepted on 22.03.2011

Int. J. Tech. 1(1): Jan.-June. 2011; Page 15-19